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ABSTRACT 



This article describes a pilot study that evaluated the 
efficacy of a computer-based, behavioral skill-building program in reducing 
aggression and improving academic performance among middle school students. 
The program is Ripple Effects' "Relate for Teens," a media rich, interactive 
application based on combining a proprietary learning system with 
research-proven strategies for shaping social behavior. The study involved 
three groups of diverse seventh and eighth graders from a New York City 
public school- -2 experimental groups of 17 students each and 1 control group 
of 23 students- -over a 12 -week period. One experimental group used the 
program as a stand-alone intervention; the other had the additional 
intervention of teacher- facilitated role-plays and discussion. The study 
examined whether the program worked in changing social behavior under either 
condition and whether it impacted academic performance. Results indicate that 
students in both experimental groups showed less aggressive behavior and more 
pro-social behavior when compared with the control group. Students in both 
experimental groups also had fewer referrals to remedial summer school. An 
unexpected finding was that students who used the program without teacher 
intervention and role-plays had greater increases in pro-social behavior and 
greater reduction in aggressive behaviors. However, the group with added 
teacher intervention had fewer remedial summer school referrals. A student 
observation assessment tracking form is appended. (MES) 
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Abstract 




Aggressive and dismptive behavior among young adolescents is a serious problem that has 
been correlated with the lack of specific social-emotional abilities that fall under the 
popular term “emotional intelligence.” For close to two decades researchers have been 
aware of effective strategies for developing those abilities. Yet these strategies have not 
been widely adopted in schools. Lack of available time is the major barrier; political issues 
and lack of teacher expertise are also factors. 

This article describes a pilot research study that evaluated the efficacy of a computer 
based, behavioral skill-building program in reducing aggression and improving academic 
performance among middle school students. The program is Ripple Effects’ Relate for 
Teens - a media rich, interactive application, based on combining a proprietary learning 
system with research proven strategies for shaping social behavior. The study involved 
three groups of diverse seventh and eighth graders from a New York City public school; 
two experimental groups of 17 students each, and one control group of 23 students, over 
a twelve-week period. One experimental group used the computer program as a stand- 
alone intervention; the other had the additional intervention of teacher facilitated role- 
plays and discussion. The study looked at two questions: “Did the Ripple Effects 
computer program work in changing social behavior under either condition?” and “Did it 
impact academic performance?” 

The results provide initial evidence that the answer to both questions is yes. Students in 
both experimental groups showed less aggressive behavior and more pro-social behavior, 
when compared with the control group. Students in both experimental groups also had 
fewer referrals to remedial summer school than the control group. 

The unexpected finding was that students who used the program without teacher 
intervention and role-plays had greater increases in pro-social behavior (77%) and greater 
reduction in aggressive behaviors (32%) than those who had additional teacher 
intervention and role-plays (32 % increase in pro-social; 22% decrease in anti social). On 
the other hand, the group with the added teacher intervention had fewer remedial summer 
school referrals (62% fewer than control group) than the one without (42% fewer than 
control group.) Because of potentially wide scale implications, more studies are needed to 
further test the efficacy of this approach. A follow-up study along with a second study 
with more subjects and observations are currently being implemented. 
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Aggressive and disruptive behavior at school is a serious concern for teachers, parents and 
students themselves. It disrupts classrooms, robs other students of valuable instruction 
time, and causes teachers to leave the profession in large numbers. Additionally, students 
who engage in aggressive behavior in classrooms often later become involved in more 
serious criminal activity (Bronfenbrenner, 1996). 

Aggressive and disruptive behavior has been linked to the lack of a specific set of social- 
emotional abilities that fall under the popular term “emotional intelligence” (Gibbs, 1995; 
Emmons & Colby, 1995). The term emotional intelligence was made popular in academic 
circles by Mayer and Salovey (see for example Salovey and Mayer, 1990 and Mayer and 
Salovey, 1993), and brought to public attention by Daniel Goleman in 1995 (Goleman, 
1995). It refers to social and emotional skills, such as self-awareness, self-motivation, 
empathy, mood management, and peer relations. Goleman, in his best selling book. 
Emotional Intelligence, reports that IQ is only a minor predictor of success in life, while 
emotional and social skills are far better predictors of success and well-being than 
academic intelligence (Goleman, 1995, 33-34). The key significance of this statement is 
that it does not focus on innate ability - the conventional understanding of intelligence - 
but upon learnable, social-emotional skills. This notion is no longer one held by select 
psychologists, but is increasingly shared by school psychologists, counselors, 
administrators and teachers all over the world. 

For close to two decades researchers have been aware of effective strategies for 
developing those abilities (Goleman, 1995, 42). Thus it is not surprising that some schools 
across the country have been integrating social and emotional skill building programs into 
their curricula. Some - though by no means most - of these programs are working. 

Several teacher and/or counselor led programs have been shown to be effective in 
reducing violence and minor aggression, improving grade point average, and increasing 
pro-social behavior (http://www.nvu.edu/education/metrocenter/initiative/modellist.html: 
http://www.hurt-free-character.com/: http://www.vouthcrimewatch-miamidade.net/) . Yet 
these strategies have not been widely adopted. 

Barriers 
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There are three major barriers preventing wide scale implementation of these programs: 

• Time constraints 

• Political constraints 

• Lack of teacher expertise 

Teachers and counselors are in a double bind situation: hours spent on social-emotional 
skill building are often seen, particularly by administrators who are hearing the call for 
increasing student achievement, as hours taken away from academic instruction. Yet not 
providing education in these areas also results in the loss of precious instruction time. 
There are simply not enough hours in the day for. most teachers and counselors to 
successfully bring students to proficiency in purely academic areas, let alone in things like 
empathy or impulse control, which are directly correlated with aggression. On the other 
hand, many teachers and counselors spend a large proportion of their counseling or 
classroom hours dealing with aggressive behavior that disrupts the learning experience for 
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everyone. So not dealing with students’ social-emotional competence also has high costs 
in terms of time (Stern, 1999). 

The second barrier is a political one. Social-emotional skill education has long been 
considered the province of families and churches - not schools. As such, it is tied up with 
issues about private morality and personal values 

(http://www.pbs.org/newshour/forum/march97/coles_3-3.httnn . School boards facing re- 
election ofteri do not want to open the Pandora’s box of dealing with emotional abilities or 
controversial social issues in the school setting 
(http ://www.montana. edu/wwwpb/home/ 1 020fam. html) . 

Finally, there is a marked lack of expertise among teachers in dealing with this area. Only 
a tiny percentage of teachers have ever received professional development themselves in 
the areas of social-emotional literacy (Goleman, 1995, 279-280). The field of social- 
emotional learning is a relatively new one, with the knowledge base changing daily. Even 
if a school or district is willing to invest in professional development, the declining 
retention rate of teachers and their pattern of frequent transfers makes it difficult, if not 
impossible, to maintain the needed level of teacher expertise. 

Conceivably, the use of electronic technology might provide the opportunity to overcome 
some of these barriers. Individualized, computer based education, could happen in free 
moments with individual students, or in group settings on a “what you need, when you 
need it” basis. The logic of a data base structure could allow local communities to pick 
and choose among topics they consider appropriate for their students. Having a changing 
knowledge base and technical expertise on-line or “in the box” could allow the constant 
incorporation of new knowledge, greatly reduce the need for instructor professional 
development, and could open up the possibility of providing social-emotional skill 
education in after school programs and other settings not staffed by professionals. 

Computer based education has already been used successfully in many areas where 
knowledge transfer is important and for skill building in technical areas, such as 
information system management and computer programming. In workplace education 
situations, it has resulted in lower costs, faster learning and greater retention rates 
(Weisinger, 1998). 

Nonetheless, the idea that computers could be effective in facilitating social-emotional 
learning is a counter-intuitive one. Emotional literacy involves much more than simple 
knowledge transfer. And the methods appropriate for the transfer of hard skills, such as 
computer programming, are very different from the strategies that have been proven 
effective for social-emotional skill education. 

Efficacy of computer based skill building 

The goal of this research study is to evaluate whether, and/or under what conditions, a 
computer based, social-emotional skill building program can be effective in positively 
impacting students’ social behavior and/or their academic performance. 

The computer based program chosen for this study is Relate for Teens, an, interactive, 
prevention program designed by Alice Ray. Ray is co-founder and CEO of Ripple Effects, 
a California software company that has received awards from the education, health and 



software industry for its groundbreaking products. She also conceived and oversaw 
development of the Second Step, when she was Executive Director of Committee for 
Children in the 1980’s. Second Step is one of a handful of violence prevention programs 
that has been clinically validated as effective (http://www.cfchildren.org/violence.htm) . 

Relate for Teens covers more than 350 issues relevant to adolescents. In addition to 
violence related topics, it includes topics such as drugs, alcohol, child abuse, anorexia, 
suicide and depression - with a goal of prevention. The software also teaches social- 
emotional skills, organized into seven core competencies: empathy, assertiveness, impulse 
control, management of feelings, decision-making ability, self-understanding and 
connection to community. These skills are correlated with reduced youth involvement in 
violence, drugs and alcohol, as well as reduced rates of the “everyday” aggression that 
makes school such a trying experience for so many teachers as well as students. 

For these reasons, this research study focused on the effectiveness of several skill building 
components of the program. Specifically, it tested the effectiveness of the program in 
reducing social aggression, through education in empathy related skills and belonging 
skills. Emerging research has also begun to show a correlation between some of these 
abilities and improved academic performance. Thus, the study also attempted to measure 
the impact of the program on academic performance. 

Relate for Teens was created using research proven strategies and research from 
education, psychology, and prevention studies. It is a database of media driven examples 
of best practices in prevention. Underpinning the program, is the Whole Spectrum 
Learning System® a proprietary, technology based learning method that developer Ripple 
Effects claims “appeals to the whole spectrum of today’s learners, with the whole 
spectrum of proven strategies, in the whole spectrum of ways today’s students prefer to 
learn” ( http://www.rippleeffects.com/) . The learning system includes a media rich 
collection of scenario based case studies, cognitive frameworks, behavioral training, 
affective motivation through true video stories, peer modeling videos, interactive writing 
exercises, transfer training opportunities with friends and family and in sports settings, 
media analysis exercises, role play instructions, interactive self profiles, and interactive, 
objective assessment exercises, all reached through a simple line of buttons at the bottom 
of a screen. Users can employ any or all of these methods in exploring any of 350+ topics. 

The program has sound to text equivalents throughout the program, increasing the chance 
that students with low reading ability, or English as a second language, can succeed with 
it. It has more than 1500 computer based writing exercises, with drag and drop word 
prompts, serving as a bridge to communication for low language level students, and type- 
your-own blanks offering open-ended opportunities for more proficient students. The 
writing entries are saved in an electronic journal, which is password protected for each 
student and encrypted to further protect student privacy. Teachers can ask students to 
print their writing exercise as an assignment, but cannot access journal entries without 
students’ permission. 




The Relate for Teens program has received seven major awards from the health, education 
and software industry, including: 

★ Best of Show, Partnerships for Health in the New Millennium, Technology Games, 
sponsored by US Department of Health and Human Services 
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★ School Winner, Software Award of Excellence, Technology & Learning Magazine 
1999-2000 

★ Silver Apple, National Educational Media Network, 1999 

★ Silver Cindy, Guidance, Values, and Career Information, 1999 

★ Silver Creative Summit Award, Educational Software, 1999 

★ Codie Finalist, Best New Education Software, 1999 

★ Codie Finalist, Best New Secondary Curriculum Software, 1999 

In addition the Ripple Effects program was named a Power User Best Bet in the National 
School Boards Association Electronic School Magazine, March 2000. It has been 
purchased for use in 80 sites in 20 states, but has yet to be evaluated for effectiveness by 
outside researchers. 




The first question to be answered in studying this program was “Does it work?” That is, 
does it result in decreases in aggressive behavior and/or increases in pro-social behavior? 
The second question to be asked was, “If it works, under what conditions is it most 
successful?” The third question to be asked was “Does it Matter? “ That is, does it 
positively impact schools primary mission, which is to attend to the academic education of 
children? 

To begin to answer these questions, the pilot study evaluated the efficacy of Relate for 
Teens in reducing anti-social behaviors, increasing pro-social behaviors, and reducing the 
need for remedial summer school among middle school students from a New York City 
public school. 

Pre and post tests; experimental and control groups 

The design involved comparing three ethnically diverse groups of middle school students 
at School of the Future, a New York City public school located in the New York City 
Board of Education Community School District 2.' Students were randomly assigned to 
the three groups at the beginning of the year. There were two experimental groups and 
one control group; the designation as experimental or control group was also randomly 
assigned. One of the two experimental groups used the computer program for 12 weeks as 
a stand-alone intervention, without teacher intervention, except for the assignment of 
lessons. The other experimental group used the computer based training program for 12 
weeks, supplemented by teacher facilitated role-plays and discussion. The control group 
received no intervention. 

Outcome measures 

The investigators used two sets of outcome variables: 

^ The authors wish to recognize the following individuals at the school who contributed to the success of 
the data collection; Barbara Leventer Luque, Director , Choice Gifted Parent Education and Maijoree 
Robbins, Director, Pupil Personnel Service at Community School District II , N.Y.C. Board of Education; 
and Jonathan Gray, Guidance Counselor, School of the Future, District II, NYC Board of Education. 
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• Discrete social behaviors, as measured by outside observers, were measured to assess 

success of the program in positively impacting social behavior. . 

• Referral rates for remedial summer school were used to measure effect of the program 
on the school’s primary educational mission. 

Hypotheses 

Investigators'hypothesized that students who were exposed to the Relate for Teens 
intervention in a planned, systematic way would show fewer anti-social behaviors and 
more pro-social behaviors, when compared with a control group at the end of the 
intervention period. 

A second hypothesis was that students who were exposed to the computerized program 
would have improved academic performance, as evidenced by fewer referrals to summer 
school than those who did not receive the program. 

A third hypothesis was that there would be differences between the experimental groups, 
with the group that received both the computer-based training and teacher facilitated role 
play interventions showing greater increases in social behavior, larger decreases in anti- 
social behavior and fewer summer school referrals than the group that received the 
computer based intervention without the role playing. 

Participants 

The participants in this study were middle school students (7“’ and 8“’ graders) from 
School of the Future, a public school in New York City. Students were randomly assigned 
to one of the three groups. Each of the three groups consisted of children that ranged in 
academic ability; each included a small number of students who attended Special 
Education classes. A total of 57 students were available in the pilot study. 

The first intervention group included 25 students in all; however, only 17 of those students 
were available for both the pre and post observations, leaving an end total of 1 7 (8 
females, 9 males). The second intervention group included 20 in all, however only 17 of 
those students were available for follow up intervention, 7 females, 10 males. There were 
23 students in the control group (1 1 females, 12 males). Observations of the control 
group were conducted during the same time period as the post observations of the two 
experimental groups. 

The sample included approximately equal numbers of African American, Hispanic, Asian, 
and Caucasian students. This sample had a greater percentage (about 75%) of non- 
Caucasian students than would a national sample. 

Methods 

Approval: A project proposal was written to the New York City Board of Education, and 
approval to conduct the project was received. 

Intervention; The Relate for Teens software intervention occurred in two forms. 
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• One of the intervention groups participated in the computer software (several times 
per week during free time over a 12-week period) and teacher assisted role-plays 
(once a week). The teacher assigned topics for the computer intervention. The 
guidance counselor assisted the role-play and the humanities teacher, who led 
discussions and organized role-playing activities according to the specific social- 
emotional competence covered that week. The teacher was available for questions 
related to the computer part of the program, but mainly the students worked 
independently. According to the guidance counselor, the role-plays resulted in many 
intense discussions among the students. 

• The other intervention group also explored topics assigned by the teacher, but did not 
participate in the role-plays; it only received the computer aspect of the intervention 
over the same 12-week period. The students in this group also used the software 
several times per week, during their free time. Although there were four computers in 
the classroom, there was one student per computer because the students did not have 
free time simultaneously. 

Students were able to follow their learning preferences in how they explored the program, 
and to do so in their free time. They were not required to go through every element in the 
whole spectrum learning system for each topic, but were required to complete the 
accompanying computer based, writing exercises and interactive assessment exercise for 
each topic. The assessment exercises consisted of two kinds of electronic games that 
parallel the structure of matching exercises and multiple-choice tests. Unlike traditional 
multiple choice tests, the assessment did not sort users into performance levels, based on 
initial answers, but rejected wrong answers until the right answer was submitted. 

According to the Ripple Effects program devejopers, the point of this approach was to use 
the process of testing as a means toward mastery, especially for those students who are 
concrete, kinesthetic learners. 

The intervention began in April and continued through to the end of the school year. 24 
topics were covered, averaging two per week in the following sequence: respect, kindness, 
paraphrasing, making space for others, giving compliments, name calling, ignoring, racial 
slurs, bullying, appreciating diversity, predicting feelings, identifying feelings, courtesy, 
body language, asking questions, expressing thanks, fighting, resolving conflicts, put 
downs, sexual harassment, stereotypes/labels, taking someone’s point of view, 
understanding feelings, and identifying with others. Students were assigned a specific topic 
to explore, but were asked to use the program during their free time. 

Observations 

The students in each of the three groups were observed by research assistants in order to 
assess their pro-social and anti-social behaviors in the classroom.^ The observations were 
conducted by graduate students in psychology and education at near by universities. 

In order to observe the greatest number of and most candid behaviors, the students were 
observed, in five-minute intervals, during independent (of the teacher) group work in 
groups ranging from two to five students. The students were observed during humanities. 



^ The authors wish to tliank Coleen McKain and Donna Klain for contributing to the training of the 
observers and collecting the data. 



art, and Spanish classes. Excluding the times when interrater reliability was being 
assessed, there was one observer per each small group of students, meaning that the 
observer was watching more than one student at a time. The observers sat as close to the 
students without interfering as possible. The students in the two experimental groups 
were observed at two different times; before the start of the intervention and at the end of 
the school year, in order to assess changes in behavior as a result of the Relate for Teens 
software. The control group was observed at the end of the school year to assess if there 
were differences among the three groups after the intervention. 

The observation tool used was the Student Observation Assessment Tracking Form (see 
Figure 1). This form lists various pro-social and anti-social behaviors that directly impact 
the school environment. The behaviors are grouped into 15 categories, encompassing the 
24 topics cited above: Many of the categories include a listing of both pro- and anti-social 
behaviors, i.e. under “respect,” both respectful and disrespectful behaviors are listed for 
observation. Examples of pro-social behaviors on the form are student takes turn in 
conversation, student gives compliments, and student uses appropriate titles. Examples 
of anti-social behaviors on the form are student responds to aggression by fighting back, 
student is defiant to teacher, and student interrupts others. Of the 53 behavioral items; 3 1 
involved pro-social behavior; 22 involved anti-social behavior. 

Each observer marked the corresponding column each time she observed a student engage 
in that behavior during a particular observation period. For analyses, the total number of 
each of the 53 behaviors for every student in the group was tallied. Then averages were 
computed for the group's 'pro-social and anti-social behavior. The Student Observation 
Assessment Tracking Form was found to be reliable. Two observers agreed 93% of the 
time when observing 24 students for five-minutes each using the 53 observation categories 
of the Student Observation Tracking Form. 

Additional Outcome Measures 

In addition to assessing the change in pro-social and anti-social behaviors using the 
Student Observation Tracking Form, investigators requested from the school, the 
following information: 

• Number of students referred to summer school for academic reasons, 

• Disciplinary referrals/suspensions^. 



These additional measures go beyond asking the question: “Did it work?” to addressing 
the more basic issue of “Does it matter?” In particular, does it impact the schools’ primary 
mission of attending to the academic education of students? 




As described above: 

• Group A received the computer intervention only (n=17) 

• Group B received computer and teacher facilitated role-play intervention (n=17) 



^ Because the participating school is relatively small and does not believe in traditional disciplinary 
action, it was not possible to gather straightforward records of that sort, such as number of times sent to 
the principal. We do have munber of suspensions, however that number is very small, total = 5. 
Therefore we will did not run analyses for that variable. 

BEST COPY AVAILABLE ^ 



• Group C served as the control group (n=23) 

Groups A and B were observed prior to the intervention using the Student Observation 
Tracking Form. Results showed no significant differences between these two groups with 
their starting behavior. All three groups were observed after the 12- week intervention 
period using the Student Observation Tracking Form. 

Since negative social behavior and truancy both normally increase toward the end of the 
school year, once having established the comparability of the two experimental groups, 
researchers focused on comparisons among the three groups at the same point in time, at 
the end of the school year. 

Findings 

The overall finding was that computer-based, behavioral education had positive impact on 
both social behavior and academic performance. 

At post-intervention, both groups that received the intervention performed more pro- 
social behaviors and fewer anti-social behaviors than the control group. Both groups that 
received the intervention had lower referral rates for summer school than the control 
group. 

Regarding social behavior: 

Mean pro-social behaviors, per student, per five-minute sample: 

• Group A (computer only) =4.67 

• Group B (computer + teacher and role plays)= 3.47 

• Group C (control group)=2.63 

Mean anti-social behaviors, per student, per 5-minute sample: 

• Group A=l. 35 

• Group B= 1.50 

• Group C=l. 91 

Although the absolute numbers were small, due to the short observation time (5 minutes 
per student), the percent rate of improvement was impressive: 

• Group A (computer only) had 77% more pro-social behavior and 32% less anti-social 
behavior than the control group 

• Group B (computer plus teacher and role play) had 32% more pro social behavior and 
22% less anti-social behavior than the control group. 

A larger group of students would be needed to claim statistical significance for the entire 
intervention. However, even with the small sample size, when broken down into the 1 5 
subscale measures, there were three areas where the differences between control and 
experimental groups reached statistical significance, and one where it came close: 

• Both intervention groups had significantly fewer (p<. 05) anti-social behaviors than the 
control group, on the “resolving conflict” subscale (items such as student tattles about 
minor matters, student responds to aggression by fighting) 
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• Both intervention groups had significantly fewer (p<. 05) anti-social behaviors than the 
control group on the “kindness” subscale (items such as student tries to embarrass 
others). 

• Group A (computer only group) had significantly more (p<. 01) pro-social behaviors 
than the other two groups on the “respect” subscale (items such as student gives 
feedback in constructive manner) . 

• Differences between groups on the “sharing” subscale, approach significance (p= .058) 
with Group A (computer only) sharing the most, followed by Group B (computer plus 
teacher), then Group C. 

Regarding academic performance 

Referral rates for remedial summer school 

• Group A 17.6% referred 

• Group B 1 1.8% referred 

• Group C 30.4% referred 

The findings about academic performance, as measured by summer school referral, were 
consistent with the original hypothesis, but not significant. Once more the percentage 
change is dramatic: 

• Group A (computer only) 42% fewer summer school referrals 

• Group B (computer + teacher and role plays) 62.5 % fewer summer school referrals 

Gender differences 

There were significant gender-correlated difference for the anti-social behaviors in the 
computer + teacher group; the girls performed significantly fewer anti-social behaviors 
(mean .94) than the boys (mean 2.1) which was significant <.01. This is consistent with 
other studies.(citation) 




While it is counter-intuitive that machines could teach social skills to urban kids, that’s 
precisely what this study suggests. It is somewhat less confounding, when one realizes that 
all of the strategies used in this program have previously been shown to work in non- 
computer based settings, using print materials, videos and live instruction 
(http ://www. nyu . edu/education/metrocenter/initiati ve/modelli st . html 1 . In one sense the 
computer just puts a broad range of traditional learning resources under one electronic 
roof 

Academic performance 

Although the numbers are small, the impact of computerized social behavior education on 
academic performance indicators is the most promising finding. The possibility that 
unassisted, student-directed use of a computerized social skill education program could 
cut summer school referral rates by more than 40% is encouraging. It’s hard to imagine a 
simpler, cheaper way to cut remedial summer school rates almost in half than to use a self- 
directed computer program that occupies students during free time. Ensuing studies need 
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to increase the number of students in the sample to statistically verify these observed 
trends. 

Social behavior 



The impacts of the intervention on social behavior are also dramatic. During the five- 
minute observation period, the students who received the computer skills building 
program for 12 weeks averaged 2 more pro-social behaviors per student and .5 less anti- 
social behaviors per student. This five-minute sampling technique was necessary to get an 
accurate count of several groups of students in a time-limited class period, without 
requiring so many observers that their presence would itself become a major factor in 
student behavior. 

However, if these changes were extrapolated to a five hour classroom day (300 minutes) it 
would equate to 120 more pro-social acts and 30 fewer anti-social acts for each student 
every day. That would mean a cumulative difference of up to 2400 more pro-social acts 
per classroom per day, and 600 fewer aggressive acts per classroom per day, more than 
triple the results with Second Step curriculum. That would create a massive change in 
school climate, on the kind of scale that could make the difference between whether a 
teacher stays in teaching or not, and whether a student skips school, or even drops out or 
not. Ensuing studies need to observe the same students for longer periods of time in a 
variety of additional schools settings, e.g., cafeterias, playgrounds, locker areas, and even 
bathrooms. 

However, the degree of social interaction in this study was skewed upward by studying 
classrooms in which students were working in groups as part of their academic training. 
Much classroom learning occurs in more structured, silent environments where social 
behavior is not a factor. Nonetheless, team based learning is not only a significant part of 
student experience; it is becoming a work place norm in the new economy, and thus will 
continue to grow in importance. 
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The anomaly 

What is most startling and invites immediate replication studies is that fact that students 
had more positive behavior changes without teacher facilitated discussion and role-plays 
than with that added element. This was not consistent with the original hypothesis, and is 
not consistent with prior research that has demonstrated that role-play is an effective, even 
necessary, method of building social skills. There are two variables to be considered, one 
is the role of the teacher as teacher; the other is the role of rehearsal or role-plays. 

Regarding the role of teacher 

• Today’s teens look to their peers, not to authority figures as a source of guidance 

• They trust computers more than they do their own parents as a source of information 
(Healy, 1998) 

The Relate program has a graffiti art style, adolescent narrators, and other teens telling 
their true stories and modeling the target behaviors. Thus, kids are getting information 
from the place they most trust, and from the peers they most want to emulate. One 
hypothesis is that the intrusion of the teacher into this sensitive area of social-emotional 
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learning puts teens on guard. They may apply a discount factor to everything that comes 
directly from an adult source, especially things that have relevance to their personal lives. 
Thus, the anomaly regarding the effect of teacher “enhancements” of the program may 
well be a developmental one that changes with younger kids. 

About the impact of role plavs 

There are several possible explanations for the fact that students, who were exposed to 
live role-plays, had fewer improvements in behavior than students who were exposed only 
to the computer program. 

It’s possible that the teacher facilitated discussions turned the emphasis away from the 
straight behavior education in the Relate program. Perhaps not every student actually 
practiced the behavior, most just talked about it. This would be consistent with prior 
research that shows that after many prevention programs, students are able to describe the 
ideal behavior, but they are not able to demonstrate it. 

It’s also possible that the actual behavior kids see modeled in the live role-play setting is 
less ideal than and/Or undermines the more precise, positive behavioral models they see 
modeled by other adolescents in the computer program. 

It’s also possible that in the group setting for live role-plays, students hold back from full 
involvement due to felt pressures to stay aloof or be cool. 

It is also possible that when a computer program accommodates individual learning 
differences, that program can be more effective than professional instruction by someone 
working with a group where individual differences cannot be accommodated. 

To obtain data about which, if any, of the above explanations explain the anomaly, 
qualitative data needs to be collected from the students to determine how they used the 
Relate program and why they behaved the way they did. 

The implications of this pilot evaluation are far reaching and invite much further study. 

The fact that the major results were positive but still lacked statistical significance is a 
function of the small sample size. Given the potential level of impact of these findings, it is 
critical that more research quickly be done to verify the trends found and explore some of 
the explanations put forth in this study. Such a effort is currently underway. 
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Figure 1 

Student Observation Assessment Tracking Form 
Date Observer Time Period 



Location 



Making Space for Others 

* 1 Student takes turn in conversation 




Group Discussions 

*2 Student waits for others to talk 
*3 Student uses open ended questions 
*4 Student makes eye contact 
*5 Student initiates problem-solving 

Resolving Conflicts 



*6 Student identifies each parties' interests 



7 Student tattles about minor matters 



*8 Student comes up with ideas to meet 
both interests 



*9 Student walks away from confrontation 



10 Student responds to aggression by 
fighting back 



* 1 1 Student includes others in solving 
conflicts 



Sharing 



*12 Student shares 



13 Student hogs class time 



14 Student hogs class resources 



Helping Others 



*15 Student offers to help others 



‘16 Student is not patronizing 



17 Student listens to the concerns of others 



Respect 



* 1 8 Student is courteous 



19 Student intrudes into others' personal 
space . 



20 Student is defiant to teacher 



2 1 Student physically intimidates other 
students 



*22 Student gives negative feedback in 
constructive manner 



*23 Student uses appropriate titles (Mr., 
Mrs., etc) 
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Behaviors 



Name Name Name Name Name Name Name 



Courtesy 



*24 Student says thank you 



25 Student forgets to say pardon me 



26 Student is late . 



27 Student interrupts others 



28 Student hogs conversation 



29 Student does not respond to question 



*30 Student responds when you greet them 
Kindness 



*3 1 Student complements others genuinely 
*32 Student says nice things to others 

33 Student antagonizes others 

34 Student tries to embarrass others 



Tolerance 



35 Student uses racial or ethnic slurs 

36 Student makes fun of people with 

disabilities 

37 Student uses sexual innuendoes or slurs 
*38 Student shows appreciation of cultural 

differences 

39 Student makes fun of low achieving 

students 

*40 Student mixes freely with students 
from other backgrounds 
*41 Student includes others in group 



Responsibility 



*42 Student admits mistakes 
43 Student refuses to apologize 



Having a Conversation 



*44 Students uses other students' preferred 
name when conversing 

45 Student interrupts others 

46 Student interrogates others 



Giving a Compliment 



*47 Student compliments others 
*48 Student praises others' behavior 



Making an Apology 

*49 Student admits being wrong 
*50 Student expresses being sorry 



Expressing Sympathy 



'‘51 Student comforts an upset peer 



Thanking 
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*52 Student thanks teacher 
*53 Student thanks other students 

* = pro-social behavior; other items = anti-social behavior 
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